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The need for afterschool programs is clear: Research and 
practice demonstrate that quality afterschool programs 
keep youth safe; support working families; and provide 
critical learning, personal development, arts, and recre¬ 
ational opportunities. New York State alone uses nearly 
$300 million in local, state, and federal funds for after- 


Services (2006), high-quality afterschool programs can 
“have significant, positive effects” on youth, yet low- 
quality programs can “fail to show positive effects or 
even have negative impacts.” Additional studies on the 
importance of afterschool program quality have been 
conducted by the Collaborative for Academic, Social, 
and Emotional Learning (Durlak & Weissberg, 2007), 
the Intercultural Center for Research in Education and 
National Institute on Out-of-School Time (Miller, 2005), 
and the Massachusetts Special Commission on After 
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school programs; it delivers public funds to organizations School and Out-of-School Time (Hall & Gruber, 2007). 

In order to advance the afterschool field, the New 


using a wide array of program models that serve many York State Afterschool Network (NYSAN) developed a 
different populations (New York State Afterschool 
Network [NYSAN], 2008). However, universal defini¬ 


tion of “quality afterschool programs” has not always 
been evident. The diversity of the afterschool field al¬ 
lows young people to have valuable, varied experiences. 
Still, the field must articulate common elements that all 
programs should incorporate into their work in order to 
maximize positive outcomes for youth. 

The critical importance of quality afterschool pro¬ 
grams in supporting youth is well documented. As re¬ 
ported by the U.S. Department of Health and Human 
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ten-element framework for program quality that has 
been widely adopted throughout the state and across the 
country. The framework hinges on the recognition that 
program quality is the best lever to realize the positive 
student outcomes that programs seek. The elements and 
corresponding indicators of quality 
in NYSANs framework provide a 
structure to promote continuous 
program improvement and profes¬ 
sional development design for out- 
of-school time programs. The 
framework has had great impact on 
how programs provide services and 
on how government agencies, inter¬ 
mediaries, and technical assistance 
specialists view program quality. 

This article follows NYSANs 
journey from developing the first it¬ 
eration of the framework through 
implementing complex strategies to promote quality 
throughout the state, ft suggests ways in which this frame¬ 
work can be useful to afterschool practitioners, technical 
assistance professionals, intermediaries, and policymak¬ 
ers nationwide. 

Program Quality: A Universal Framework 

The program quality framework was developed over 
two years and published in 2005 by NYSAN, a public- 
private partnership and one of 39 statewide afterschool 
networks (National Network of Statewide Afterschool 
N etworks, n. d.). NYSAN’s Quality Assurance Committee, 
a group of statewide afterschool experts, developed the 
framework with input and feedback from a larger group 
of afterschool practitioners and national experts. 
Because NYSAN is a partnership of multiple stakehold¬ 
ers, the quality framework reflects the consensus of a 
wide range of partners, including state agencies, large 
intermediaries, and small community-based organiza¬ 
tions. The framework transcends a program’s model, 
geography, and host setting, yet provides a detailed 
structure for what high-quality programs should aim to 
achieve. The framework can be used by school- and 
community-based programs—licensed or legally ex¬ 
empt—whether they operate before school, after school, 
or during the summer. 

NYSAN structured the framework around ten es¬ 
sential elements of program quality, each of which is 
defined by a list of specific quality indicators. The 
framework incorporates national and local standards 
and research, including the National AfterSchool 


Association standards (National School-Age Care 
Alliance, 1998), with local work done across the coun¬ 
try, such as frameworks developed in Baltimore 
(Baltimore Safe and Sound Campaign, 1999), Boston 
(Achieve Boston, 2003), and Los Angeles (Freeman & 
Redding, 1999). 

NYSAN partners considered fac¬ 
tors unique to New York; they subse¬ 
quently added concepts derived from 
the New York State School-Age Child 
Care regulations (NYS Office of 
Children and Family Services, 2005) 
and feedback from New York-based 
program providers. The resulting 
framework includes a full spectrum 
of criteria, including point-of-service, 
administrative, and management ele¬ 
ments. Some of the indicators are ob¬ 
servable, while others might be writ¬ 
ten into policies or documented in program records. The 
ten essential elements of program quality are: 

• Environment and climate 

• Administration and organization 

• Relationships 

• Staffing and professional development 

• Programming and activities 

• Linkages between school and afterschool 

• Youth participation and engagement 

• Parent, family, and community partnerships 

• Program growth and sustainability 

• Measuring outcomes and evaluation 

A recent meta-analysis of eight research studies and 
existing program quality frameworks (Palmer, Anderson, 
& Sabatelli, 2009) affirmed that the field is beginning to 
agree on what makes a high-quality program. 

Six domains—supportive relationships, intentional 
programming, strong community partnerships, pro¬ 
motion of youth engagement, physical safety, and con¬ 
tinuous program improvement—represent clear 
points of convergence across the various definitions of 
program quality. The held is reaching consensus re¬ 
garding what aspects of program quality are important 
and how these dimensions of program quality fit into 
the overall picture of afterschool programming. 
(Palmer, Anderson, & Sabatelli, 2009, p. 9) 

Though the language and structure of the categories 
is nuanced, NYSAN’s quality framework includes each of 
these domains; it is relevant to and aligned with the most 
current research on afterschool program quality. 
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Strategies for Promoting High-Impact 
Afterschool for All 

Highlighted below are the tools, strategies, and activi¬ 
ties that agencies and organizations have employed to 
support program improvements using the NYSAN 
quality framework. 

Program Quality Self-Assessment Tool 
NYSAN member organizations realized that defining 
quality and its component parts was but one step in mak¬ 
ing a contribution to the field. Therefore, NYSAN devel¬ 
oped the Program Quality Self-Assessment (QSA) Tool. 
Use of the QSA Tool is a critical component of many 
programs’ quality improvement strategies. The self- 
assessment process uses the quality 
framework to provide structure for 
afterschool professionals to reflect 
on their practice through dialogue 
and to own the process of continu¬ 
ously improving their programs. 

Unlike from many other assessment 
tools, the QSA Tool is designed to be 
used by program staff and other 
stakeholders, including youth, fam¬ 
ily members, and school and com¬ 
munity partners, without an exter¬ 
nal observer. Programs may use the QSA Tool to assess 
their program along all ten elements of program quality 
at one time, or they may use parts of the QSA Tool over 
several weeks or even throughout the program year. 

In August and September 2009, NYSAN used a web- 
based instrument to survey New York State afterschool 
providers about their use of the QSA Tool (NYSAN, 
2009). Respondents included 106 program providers 
from all regions of the state, including large and small 
programs in rural, suburban, and urban communities. 
Users of the QSA Tool reported numerous benefits to 
their programs. First and foremost, the self-assessment 
successfully guided users through the processes of both 
assessing quality and creating an action plan that fosters 
a shared sense of ownership and accountability among 
program stakeholders, while also building consensus 
about what constitutes a high-quality program. Program 
providers also reported unanticipated results of use of 
the tool, including attracting funding and improving re¬ 
lationships between programs and schools. Eighty-three 
percent of QSA Tool users reported reaping benefits from 
use of the QSA Tool, and 75 percent reported one or 
more distinct changes in their program after using the 
QSA Tool for self-assessment. Moreover, 55 percent of 


respondents identified quality improvements in their 
program as a result their self-assessment. 

Loretta McCormick oversees the Creating Rural 
Opportunities Partnership (CROP), a consortium of six¬ 
teen rural school districts that collectively aim to provide 
afterschool environments that are safe and supportive; 
promote student achievement; foster learning through 
personal, social, and positive youth development oppor¬ 
tunities; and engage parents and guardians in experi¬ 
ences that foster a greater connection with their school 
and community. McCormick said that CROP, a 21st 
Century Community Learning Center (21st CCLC) 
grantee, uses the QSA Tool to identify strengths and areas 
in need of improvement across multiple program sites, 
many of which are separated by long 
distances. CROP uses the results to 
focus its biannual professional de¬ 
velopment offerings on topics iden¬ 
tified through the self-assessment 
process. Having a common assess¬ 
ment tool allows each program to 
work toward the same goals and 
share a common vision despite geo¬ 
graphic distance (NYSAN, 2007). 

In New York City, Doreen Teh 
of the Child Center of New York’s 
program at PS. 24 asserts that the self-assessment process 
allows her to develop stronger partnerships and improve 
the sustainability of her program: 

[The QSA Tool] has strengthened the communica¬ 
tion between the afterschool program and the school; 
all staff members have a clear understanding of pro¬ 
gram goals and can identify real opportunities for 
working together. We are speaking the same lan¬ 
guage and are more focused and intentional in our 
practice. It has created a common framework to 
guide our work. (NYSAN, 2007, p. 32) 

Technical Assistance 

In addition to the QSA Tool itself, programs have access 
to the QSA Tool User’s Guide, which provides practical 
strategies and examples from afterschool practitioners on 
best practices in self-assessment and program improve¬ 
ment. The user’s guide includes information on begin¬ 
ning a self-assessment, engaging stakeholders in the pro¬ 
cess, and using assessment findings to create an action 
plan for addressing areas in need of improvement. 

Moreover, NYSAN developed an online version of 
the user’s guide, which provides direct links to dozens of 
resources organized around the ten elements of program 
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FOR MORE INFORMATION 


The second edition of the NYSAN quality 
framework and QSA Tool is being 
released in 2010. For more information 
on using or adapting the quality 
framework, user's guide, or QSA Tool, 
please contact NYSAN at 646-943-8670 or 
info@nysan.org. 


quality. This user’s guide helps afterschool practitioners 
assess their programs and make feasible, effective changes 
to improve program quality. The user’s guide booklet and 
website are available, free of charge, at www.nysan.org. 

Additional supports include conference workshops, 
trainings, and program supports designed around the 
quality framework. Two annual statewide conferences, 
designed for 21st CCLC grantees but attended by a wide 
audience, are structured using the ten elements of pro¬ 
gram quality. Several NYSAN partner organizations, in¬ 
cluding The After-School Corporation and the Partnership 
for After School Education, provide program quality ele¬ 
ments from the quality framework in their training 
menus. Regional afterschool networks throughout the 
state also use the quality framework to offer professional 
learning opportunities on program quality. Collectively, 
these supports are part of a growing consensus across the 
state regarding building capacity for quality program¬ 
ming. 

The New York State Education Department (NYSED) 
contracts with six Regional Student Support Services 
Centers; each of the centers employs a regional coordina¬ 
tor who is trained in and has experience with the QSA 
Tool. The regional coordinators often facilitate programs’ 
use of the QSA Tool or design program supports based 
on the results of their self-assessments. At the Hudson 
Valley Student Support Services Center, Regional 
Coordinator Tammy Rhein shapes region-wide technical 
assistance and professional development plans around 
common areas in need of improvement as recognized 
through local programs’ self-assessment results (personal 
communication, July 6, 2009). 

Carol Marshall, a teacher trainer with the Mid-State 
Student Support Services Center, facilitates conversa¬ 
tions about quality by starting with two elements identi¬ 
fied by a program’s leaders as most relevant to the success 
and sustainability of their work. According to Marshall, 
one of the most useful purposes of the quality framework 
is to help programs become familiar with best practices. 


Marshall noted, “I encourage them to learn and use the 
language when writing objectives for their activities, or 
for applying for grants that require a comprehensive, re¬ 
search-based approach to making a program successful 
and sustainable. The QSA Tool is an effective resource for 
so much more than self-assessment” (personal commu¬ 
nication, July 6, 2009). 

Statewide Institute for Public Agencies 

As part of a long-term effort to build a coordinated, state¬ 
wide system to support afterschool programs, NYSAN 
led a two-year professional development experience for 
the program managers of the major public funding 
streams for afterschool programs in which the QSA Tool 
was a central component. 

Participating program managers represented staff 
from NYSED and NYS Office of Children and Family 
Services, as well as the New York City Departments of 
Education and of Youth and Community Development. 
The quality framework provided structure for events 
throughout the two-year initiative. NYSAN conducted a 
needs assessment and provided training and support in 
the areas of program quality requested most by institute 
participants. As a result, all of the state and city agency 
staff members who oversee large afterschool grants are 
equipped to use the quality framework and QSA Tool 
with the hundreds of publicly funded programs they 
oversee and support. 

Statewide Policy Development 
Approaching program quality through policy develop¬ 
ment has proved to be an effective strategy to support the 
development and sustainability of high-impact after¬ 
school programs. NYSED has adopted the quality frame¬ 
work in two ways: requiring that agencies seeking 21st 
CCLC funds design their programs around the ten ele¬ 
ments of quality and subsequently requiring grantees to 
use the QSA Tool twice each year. These requirements 
are written into the 21st CCLC request for proposals as 
well as other NYSED documents (NYS Education 
Department, 2009). In New York City, the Department of 
Youth and Community Development used the self-as¬ 
sessment tool to monitor grantees of the agency’s Out-of- 
School Time Initiative, the largest municipally funded 
out-of-school time system in the nation. The agency tells 
programs that they will be held accountable to the QSA 
standards, thereby encouraging them to use the QSA 
Tool to design programs, maintain program quality, and 
fulfill agency mandates. By using these policy strategies, 
New York State is building a highly-effective afterschool 
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system based on common quality standards in which con¬ 
tinuous improvement and self-assessment are linked 
with reporting and accountability requirements. 

National Use of the QSA Tool 

As a result of its relevance to a myriad of programs, New 
Yorks quality framework has been adapted in several 
states including Missouri, California, Ohio, and New 
Mexico, each of which used NYSAN’s advice and experi¬ 
ence in creating their frameworks and self-assessment 
tools. In some cases, states used the framework and QSA 
Tool and simply edited them to reflect local regulations 
and promising practices. For some states, the statewide 
afterschool network led the process, while in others the 
state education agency or local groups took the lead and 
have been successful in promoting wide adoption of their 
new tool. Data collected through the aforementioned 
survey (NYSAN, 2009) and anecdotes from across the 
nation suggest that the framework and the self-assessment 
tool are a powerful pair, providing 
both research-based standards and 
an accessible instrument to help 
programs meet them. 

Lessons Learned 

NYSAN has documented the devel¬ 
opment of the quality framework 
and QSA Tool to disseminate infor¬ 
mation about the process to other 
states and organizations. NYSAN 
has found that raising awareness 
and providing training in the use of the quality frame¬ 
work and the self-assessment tool are critical to promot¬ 
ing wide adoption of both elements. In our survey, 37 
percent of respondents had never used the QSA Tool, and 
40 percent reported lack of understanding as a barrier to 
its use in their programs. Many such respondents report¬ 
ed that education and training would support their pro¬ 
grams’ use of the QSA Tool (NYSAN, 2009). 

NYSAN has also learned that the quality framework 
and QSA Tool must continuously evolve in response to 
new research and changes in the field. The NYSAN 
Quality Assurance Committee is currently revisiting all 
quality indicators to be sure they remain essential to 
program quality. Perhaps more importantly, the com¬ 
mittee has discussed adding several indicators that ei¬ 
ther provide new concepts or address aspects of pro¬ 
gram quality more explicitly. For example, one new 
indicator would address a current priority for the after¬ 
school and early childcare communities by specifically 


linking afterschool and early learning programs and 
policies. Another planned addition will incorporate 
language regarding inclusion of youth of all levels of 
ability. Additionally, several new indicators will be added 
to increase alignment between the QSA Tool and other 
quality assessment tools, such as the Youth Program 
Quality Assessment (Weikart Center for Youth Program 
Quality, 2005). 

Achieving High-Impact Afterschool for All 

Access to afterschool programs alone is insufficient; qual¬ 
ity counts in ensuring that youth have access to support¬ 
ive, effective afterschool programming. The field is rec¬ 
ognizing this tenet more and more, as evidenced by a 
growing investment in quality assessment. As described 
by the Forum for Youth Investment (Yohalem & Wilson- 
Ahlstrom, 2009): 

From a research perspective, more evaluations are 
including an assessment of program quality and 
many have incorporated setting- 
level measures (where the object of 
measurement is the program, not 
the participants) in their designs.... 
At the policy level, decision-makers 
are looking for ways to ensure 
that resources are allocated to pro¬ 
grams likely to have an impact.... 
At the practice level, programs, or¬ 
ganizations and systems are look¬ 
ing for tools that help concretize 
what effective practice looks like 
and allow practitioners to assess, reflect on and im¬ 
prove their programs, (p. 6) 

The NYSAN quality framework and QSA Tool have 
moved programs in New York State closer to meeting 
quality standards and have created a stronger culture of 
continuous improvement in local and statewide after¬ 
school systems. 
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